1. Field of the Invention
The present invention relates to an image search apparatus and an image search method, and a storage medium thereof in which a feature amount for image search is inputted and an image is searched under the search conditions for the feature amount.
2. Description of the Related Art
At present, a digital camera or a portable telephone with camera has widespread, whereby the chance for taking a digital image increases. On the other hand, the storage media using a non-volatile RAM such as a CF (Compact Flash (registered trademark)) card, an SD (Secure Digital (registered trademark)) card or a memory stick (registered trademark) tends to have a larger capacity. Along with this, a greater number of images are stored in the media. Also, there is arranged a use environment in which the taken image is read into an owned PC (Personal Computer), and stored in an HDD (Hard Disk Drive) unit for browsing or printing.
Recently, the applications for supporting the image browsing or image selection operation as image management software are increasing. Some applications provide a file management function using an Exif (Exchangeable Image File Format) (registered trademark) standardized by JEIDA.
The Exif provides a function of storing the shooting information in a format of Exif Tag. With this Exif tag, the information such as shooting time, exposure amount, focal length and WB (White Balance) can be known later.
For example, a UI (User Interface) is provided through which the image can be simply searched according to the shooting time by sorting the images managed by the shooting time and linking them with the calendar.
Also, the image management software provides the user with another function of adding arbitrary tag information. With such image management software, the user can add a keyword to one or more images selected by the user. The keyword may be any string. For example, a “wedding ceremony” or an “entrance ceremony” may be inputted as the keyword.
In this manner, by adding beforehand the keyword to the image, the user can perform the search by inputting the keyword such as wedding ceremony or entrance ceremony.
However, as the number of images increases, the user has more load to add the keywords of various strings to the images, whereby it is difficult to add the keywords that can classify the images simply.
Also, when the image is searched by keyword, the keyword for specifying the image may be long or a combination of plural strings.
For example, consider a case of searching and finding an image 1001 as shown in FIG. 10. The image 1001 has the keywords of “Mr. A” and “Mr. B” appended as the Tag information 1002. But the other image 1003 has also the same keywords appended as the Tag information 1004.
In such a state, if the search is made according to the keywords of “Mr. A” and “Mr. B”, two images 1001 and 1003 are obtained as the search results.
However, actually, the image 1001 has Mr. A on the left side of this image and Mr. B on the right side, which are of the same size, while the image 1003 has two persons in different arrangement and face size.
Accordingly, when only the image 1001 is searched, it is required to add the detailed information regarding the composition such as “Mr. A”, “Mr. B”, and “Mr. A on the right side”. However, it is necessary that the representation regarding the composition is systematically added even personally at the time of appending the tag. Also, when another person makes the search, the analysis itself for the keyword and the search itself can not be performed unless the way of appending the keyword is unified.
Further, if information such as the size of each face is not added to the tag, it is difficult to limit the search result.
In this manner, to search the image by keyword, it is sought to add the systematical keyword and add the detailed composition information. It is difficult for the general user to make such things steadily.
As described above, the invention for solving a problem of searching the image by keyword was disclosed in Japanese Patent Laid-Open No. 2000-076268 (hereinafter referred to as a document 1) and Japanese Patent Laid-Open No. 2001-084274 (hereinafter referred to as a document 2), for example.
The document 1 disclosed a method in which the feature amount of the image drawn by the user is obtained and the image having the high degree of similarity with the feature amount is displayed as the search result. With this method, the user can input the search conditions as illustration. Accordingly, the method can give the higher representation ability for searching than the keyword.
Also, the document 2 disclosed a method for searching the image by inputting a simple graphic such as circle, triangle or rectangle and obtaining the degree of agreement with the subject in the image.
With both the techniques as disclosed in the documents 1 and 2, the feature amount of the illustration image inputted by the user is obtained, and the image having higher degree of similarity with the feature amount is searched from an image database (hereinafter denoted as an image DB) to display the search results.
The images taken by a photographing apparatus such as a digital camera include the horizontally shot image 1001 and the vertically shot image 1003, as shown in FIG. 10. Especially in the case of the vertically shot image 1003, the user may rotate the photographing apparatus by 90° clockwise or counterclockwise in taking the image.
In the normal case, the shooting data stored as image data is generally stored as horizontally oriented data, irrespective of the orientation at the time of shooting, because the image pickup device for the photographing apparatus is mostly horizontally oriented. Accordingly, the vertically shot image 1003 is stored as the image rotated by +90° or −90° as the horizontally oriented image like the horizontally shot image 1001. FIG. 10 shows the image 1003′ rotated by +90° clockwise.
Accordingly, when the image search is performed by the method as disclosed in the document 1 or 2, there is a problem that the desired result can not be obtained only by directly applying the conditions inputted by the user to the image data.
That is, even if the search conditions are inputted at an aspect (length-to-width) ratio of the image limited in the case of searching the image where two persons sit side by side, the images 1001 and 1003′ can not be searched at the same time. Also, when the image 1003′ is rotated into alignment in the upper, lower, left and right, and stored as the image 1003, these three images can not be searched.
With these conventional methods, when the images taken under various conditions such as plural aspect ratios, horizontal shooting and vertical shooting are collectively selected and searched, the search conditions adaptable to all the images can not be specified, whereby there was a problem that the search can not be made as intended.